Minimum Message Length Ridge Regression for Generalized Linear Models

This paper introduces an information theoretic model selection and ridge parameter estimation criterion for generalized linear models based on the minimum message length principle. The criterion is highly general in nature, and handles a range of target distributions, including the normal, binomial, Poisson, geometric and gamma distributions. Estimation of the regression parameters, the ridge hyperparameter and the set of covariates associated with targets is all performed within the same framework by minimisation of the message length. Experiments on simulated and real data suggest that the criterion is competetive with, and often superior to, the corrected Akaike information criterion in terms of both parameter estimation and model selection tasks.